{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Garimpagem de Dados\n",
    "\n",
    "## Aula 4 - Exercídio de Classificação com kNN\n",
    "13/10/2017"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Aluno: Marcos Felipe de Menezes Mota - 354080"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset:** Titanic: Machine Learning from Disaster\n",
    "\n",
    "https://www.kaggle.com/c/titanic/data\n",
    "\n",
    "Partindo da aula passada:\n",
    "\n",
    "1. Atualizar a função que mede a distância euclidiana para o pacote do scikit-learn \n",
    "\n",
    "2. Implementar uma função que selecione os k vizinhos mais próximos (k > 1)\n",
    "\n",
    "3. Implementar uma função que recebe os k vizinhos mais próximos e determinar a classe correta\n",
    "\n",
    "4. Transformar as features categoricas em numéricas (tip: pandas ou scikit-learn)\n",
    "\n",
    "5. Analisar a necessidade de normalizar as features numéricas (tip: pandas ou scikit-learn)\n",
    "\n",
    "6. Selecionar as features baseada na correlação (tip: pandas)\n",
    "\n",
    "7. Separar o dataset em treino (75%) / teste (25%) / validação (10% do treino)\n",
    "\n",
    "4. Execute o classificador para 30 k's pulando de 4 em 4 e apresente todas as acurácias utilizando o dataset de validação (Qual o melhor k?) [plotar um gráfico com os resultados]\n",
    "\n",
    "5. Executar o classificador para o melhor k encontrado utilizando o dataset de teste e apresentar um relatório da precisão (tip: scikit-learn) [plotar um gráfico com os resultados]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn import datasets\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.neighbors import DistanceMetric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class KNNClassifier(object):\n",
    "    def __init__(self):\n",
    "        self.X_train = None\n",
    "        self.y_train = None\n",
    "        self.distance = \"euclidian\"\n",
    "        self.k = 1\n",
    "\n",
    "    def f_distance(self, a, b):\n",
    "        dist = DistanceMetric.get_metric(self.distance)\n",
    "        matDist = dist.pairwise([a,b])\n",
    "        return matDist[0,-1]\n",
    "\n",
    "    def closest(self, row):\n",
    "        \"\"\"\n",
    "        Retorna a classe respondente ao ponto mais próximo do dataset de treino.\\\n",
    "        É um exemplo de implementação do kNN com variável\n",
    "        \"\"\"\n",
    "        dists = [self.f_distance(row, item) for item in self.X_train]\n",
    "        neighbors = sorted(dists)[:self.k]\n",
    "        nei = [dists.index(x) for x in neighbors]\n",
    "        votes = self.y_train[nei]\n",
    "        label = np.argmax(np.bincount(votes))\n",
    "        return label\n",
    "\n",
    "    def fit(self, training_data, training_labels, k=1, distance=\"euclidian\"):\n",
    "        self.X_train = training_data\n",
    "        self.y_train = training_labels\n",
    "        self.k = k\n",
    "        self.distance = distance\n",
    "\n",
    "    def predict(self, to_classify):\n",
    "        predictions = []\n",
    "        for row in to_classify:\n",
    "            label = self.closest(row)\n",
    "            predictions.append(label)\n",
    "        return predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d_train = pd.read_csv(\"train.csv\")\n",
    "d_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Transformar dados categoricos e limpando algumas colunas**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>1</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>146</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>81</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>146</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>55</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>146</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name  Sex   Age  SibSp  Parch  \\\n",
       "0                            Braund, Mr. Owen Harris    1  22.0      1      0   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...    0  38.0      1      0   \n",
       "2                             Heikkinen, Miss. Laina    0  26.0      0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)    0  35.0      1      0   \n",
       "4                           Allen, Mr. William Henry    1  35.0      0      0   \n",
       "\n",
       "             Ticket     Fare  Cabin  Embarked  \n",
       "0         A/5 21171   7.2500    146         3  \n",
       "1          PC 17599  71.2833     81         0  \n",
       "2  STON/O2. 3101282   7.9250    146         3  \n",
       "3            113803  53.1000     55         3  \n",
       "4            373450   8.0500    146         3  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Imputação de dados\n",
    "d_train.Embarked = d_train.Embarked.fillna(\"None\")\n",
    "d_train.Cabin = d_train.Cabin.fillna(\"None\")\n",
    "d_train.Age = d_train.Age.fillna(d_train.Age.mean())\n",
    "\n",
    "#Transformandos categoricos \n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "encoder = LabelEncoder()\n",
    "d_train['Sex'] = encoder.fit_transform(d_train.Sex)\n",
    "d_train['Cabin'] = encoder.fit_transform(d_train.Cabin)\n",
    "d_train['Embarked'] = encoder.fit_transform(d_train.Embarked)\n",
    "\n",
    "d_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Usando pandas para mostrar matrix de correlação e remover as caracteristicas mais correlacionadas**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.005007</td>\n",
       "      <td>0.035144</td>\n",
       "      <td>0.042939</td>\n",
       "      <td>0.033207</td>\n",
       "      <td>0.057527</td>\n",
       "      <td>0.001652</td>\n",
       "      <td>0.012658</td>\n",
       "      <td>0.035197</td>\n",
       "      <td>0.009305</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>0.005007</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.338481</td>\n",
       "      <td>0.543351</td>\n",
       "      <td>0.069809</td>\n",
       "      <td>0.035322</td>\n",
       "      <td>0.081629</td>\n",
       "      <td>0.257307</td>\n",
       "      <td>0.253658</td>\n",
       "      <td>0.174963</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>0.035144</td>\n",
       "      <td>0.338481</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.131900</td>\n",
       "      <td>0.331339</td>\n",
       "      <td>0.083081</td>\n",
       "      <td>0.018443</td>\n",
       "      <td>0.549500</td>\n",
       "      <td>0.682176</td>\n",
       "      <td>0.197493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>0.042939</td>\n",
       "      <td>0.543351</td>\n",
       "      <td>0.131900</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.084153</td>\n",
       "      <td>0.114631</td>\n",
       "      <td>0.245489</td>\n",
       "      <td>0.182333</td>\n",
       "      <td>0.095991</td>\n",
       "      <td>0.106395</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>0.033207</td>\n",
       "      <td>0.069809</td>\n",
       "      <td>0.331339</td>\n",
       "      <td>0.084153</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.232625</td>\n",
       "      <td>0.179191</td>\n",
       "      <td>0.091566</td>\n",
       "      <td>0.234912</td>\n",
       "      <td>0.034883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>0.057527</td>\n",
       "      <td>0.035322</td>\n",
       "      <td>0.083081</td>\n",
       "      <td>0.114631</td>\n",
       "      <td>0.232625</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.414838</td>\n",
       "      <td>0.159651</td>\n",
       "      <td>0.043525</td>\n",
       "      <td>0.068043</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>0.001652</td>\n",
       "      <td>0.081629</td>\n",
       "      <td>0.018443</td>\n",
       "      <td>0.245489</td>\n",
       "      <td>0.179191</td>\n",
       "      <td>0.414838</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.216225</td>\n",
       "      <td>0.028179</td>\n",
       "      <td>0.032517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.012658</td>\n",
       "      <td>0.257307</td>\n",
       "      <td>0.549500</td>\n",
       "      <td>0.182333</td>\n",
       "      <td>0.091566</td>\n",
       "      <td>0.159651</td>\n",
       "      <td>0.216225</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.502569</td>\n",
       "      <td>0.246359</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cabin</th>\n",
       "      <td>0.035197</td>\n",
       "      <td>0.253658</td>\n",
       "      <td>0.682176</td>\n",
       "      <td>0.095991</td>\n",
       "      <td>0.234912</td>\n",
       "      <td>0.043525</td>\n",
       "      <td>0.028179</td>\n",
       "      <td>0.502569</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.232192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>0.009305</td>\n",
       "      <td>0.174963</td>\n",
       "      <td>0.197493</td>\n",
       "      <td>0.106395</td>\n",
       "      <td>0.034883</td>\n",
       "      <td>0.068043</td>\n",
       "      <td>0.032517</td>\n",
       "      <td>0.246359</td>\n",
       "      <td>0.232192</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             PassengerId  Survived    Pclass       Sex       Age     SibSp  \\\n",
       "PassengerId     1.000000  0.005007  0.035144  0.042939  0.033207  0.057527   \n",
       "Survived        0.005007  1.000000  0.338481  0.543351  0.069809  0.035322   \n",
       "Pclass          0.035144  0.338481  1.000000  0.131900  0.331339  0.083081   \n",
       "Sex             0.042939  0.543351  0.131900  1.000000  0.084153  0.114631   \n",
       "Age             0.033207  0.069809  0.331339  0.084153  1.000000  0.232625   \n",
       "SibSp           0.057527  0.035322  0.083081  0.114631  0.232625  1.000000   \n",
       "Parch           0.001652  0.081629  0.018443  0.245489  0.179191  0.414838   \n",
       "Fare            0.012658  0.257307  0.549500  0.182333  0.091566  0.159651   \n",
       "Cabin           0.035197  0.253658  0.682176  0.095991  0.234912  0.043525   \n",
       "Embarked        0.009305  0.174963  0.197493  0.106395  0.034883  0.068043   \n",
       "\n",
       "                Parch      Fare     Cabin  Embarked  \n",
       "PassengerId  0.001652  0.012658  0.035197  0.009305  \n",
       "Survived     0.081629  0.257307  0.253658  0.174963  \n",
       "Pclass       0.018443  0.549500  0.682176  0.197493  \n",
       "Sex          0.245489  0.182333  0.095991  0.106395  \n",
       "Age          0.179191  0.091566  0.234912  0.034883  \n",
       "SibSp        0.414838  0.159651  0.043525  0.068043  \n",
       "Parch        1.000000  0.216225  0.028179  0.032517  \n",
       "Fare         0.216225  1.000000  0.502569  0.246359  \n",
       "Cabin        0.028179  0.502569  1.000000  0.232192  \n",
       "Embarked     0.032517  0.246359  0.232192  1.000000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_matrix = d_train.corr().abs()\n",
    "corr_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Deletando colunas desnecessarias e com alta correlação\n",
    "del d_train['Name']\n",
    "del d_train['PassengerId']\n",
    "del d_train[\"Fare\"]\n",
    "del d_train[\"Cabin\"]\n",
    "del d_train[\"Ticket\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass  Sex   Age  SibSp  Parch  Embarked\n",
       "0         0       3    1  22.0      1      0         3\n",
       "1         1       1    0  38.0      1      0         0\n",
       "2         1       3    0  26.0      0      0         3\n",
       "3         1       1    0  35.0      1      0         3\n",
       "4         0       3    1  35.0      0      0         3"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y = d_train.Survived.values\n",
    "del d_train[\"Survived\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "scaler = MinMaxScaler()\n",
    "X = scaler.fit_transform(d_train[:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_tt, X_valid, y_tt, y_valid = train_test_split(X, y, test_size=0.1, random_state=2)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_tt, y_tt, test_size=0.25, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "list_of_ks = range(1, 120, 4)\n",
    "knn = KNNClassifier()\n",
    "scores = []\n",
    "from sklearn.metrics import accuracy_score, classification_report\n",
    "\n",
    "for k in list_of_ks:\n",
    "    knn.fit(X_train, y_train, k=k, distance='euclidean')\n",
    "    result = knn.predict(X_test)\n",
    "    score = accuracy_score(y_pred = result, y_true = y_test)\n",
    "    scores.append(score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.74129353233830841, 0.80099502487562191, 0.82587064676616917, 0.8159203980099502, 0.83582089552238803, 0.85074626865671643, 0.85074626865671643, 0.845771144278607, 0.84079601990049746, 0.845771144278607, 0.845771144278607, 0.845771144278607, 0.845771144278607, 0.845771144278607, 0.845771144278607, 0.845771144278607, 0.845771144278607, 0.84079601990049746, 0.8308457711442786, 0.8308457711442786, 0.8308457711442786, 0.82587064676616917, 0.82587064676616917, 0.82587064676616917, 0.82587064676616917, 0.82587064676616917, 0.82587064676616917, 0.82587064676616917, 0.82587064676616917, 0.82587064676616917]\n"
     ]
    }
   ],
   "source": [
    "print(scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAHEZJREFUeJzt3X+cVXWdx/HXZ/ghIiIWIxAgg2mu\nYLbkiL/QSsUUC/xRG2StuhZtiaymrj8qQ5JNM9PcNBfJtW0NYi0LHQxN8ccoCkOICiwKqICgDhar\naILDfPaP7xm5ztyZe2fmzj33nvN+Ph7zuHPPPefcz/HKe879nu/5fs3dERGRdKiIuwARESkehb6I\nSIoo9EVEUkShLyKSIgp9EZEUUeiLiKSIQl9EJEUU+iIiKaLQFxFJke5xF9Bc//79vaqqKu4yRETK\nytKlS7e4e2Wu9Uou9Kuqqqirq4u7DBGRsmJmL+eznpp3RERSRKEvIpIiCn0RkRRR6IuIpIhCX0Qk\nRRT6perOO6GqCioqwuOdd8ZdkYgkQMl12RRCwE+eDO+8E56//HJ4DnDmmfHVJSJlT6Ffir7znV2B\n3+Sdd9h63nf4yersod+zJ5x9NgwZ0vXliUj5slKbI7e6utpTf3NWRQVk+VwaMbpbY9ZN3GHQILj3\nXvjkJ7u6QBEpNWa21N2rc62nNv1StO++WRdXDNuXxkay/jz7bDjbP+YYmDevyPWKSNlQ6JeiGTPY\n0b33B5f17g0zZrS6ycEHw5NPwsiRcOqpcOONWb8siEjKKfRL0I4vnsnU3Wfy+u7DwAyGDYOZM3Ne\nxB04EB5+GE4/HS68EKZMgYaG4tQsIuVBoV+C7r0X/uOtM6m766XQdvPSS3n32undG+bOhX/9V7jl\nFhg/Ht58s0vLFZEyotAvQbNmweDB8NnPdmz7igq49trw5eD++2HMGNiwobA1ikh5UuiXmA0bYMEC\nOOcc6Natc/v6+tfhvvtCN//DD4elSwtTo4iUL4V+ibnjjtCi80//VJj9jR0LTzwRevYceyz84Q+F\n2a+IlCfdnFVCGhvh9tvhhBNg+PDC7XfkSHjqqdC+f9pp8OMfhwu9ZoXZ/5NPwj33FGZfSbbbbuHb\n16BBcVciaabQLyEPPRSu2f7wh4Xf94ABsHAhnHUWXHQRvPAC/Pu/Q/dO/h/wy1+GINu5M1xLkNY1\nNMBtt0FNDRxySNzVSFop9EvIrFnwoQ+FfvZdoXdv+M1v4IorwoXeF18MPX369m3/vhob4corw60D\nxx8Pd90F/foVvuYkWbYMPvc5OPro8N/95JPjrkjSSOdmJeKNN+Duu+GrX4VevbrufSoq4Jprwhnn\ngw+Gnj3r17dvH+++C1/+cgj8c88NF4sV+LmNGgWLF8P++4fwv+WWuCuSNFLol4j//m/YsSOEaDF8\n7WshrNevDz178h3uqL4ejjsufGNo+uPRo0fX1pokgwfDY4/BuHFw3nnh2srOnXFXJWmi0C8B7qFp\nZ/Ro+PjHi/e+J5wQevb06hV69tx9d9vrr1oV/kAsWwb/8z9w6aWFuxicJn36wO9/D1OnhuEyTj8d\ntm2LuypJC4V+CVi8GJ57Lpx9F9uIEaH3zSc+AWecEXr2ZBuz56GH4Mgj4e23w1APX/hC0UtNlG7d\n4Kc/DRfT7703/NHdtCnuqiQNFPolYNascJH1S1+K5/0HDAih/sUvwiWXwD//M7z33q7Xb7893B08\neHDo+nn44fHUmURTpoRRUV94Ifx3Xb487ook6fIKfTM7ycxWm9kaM7ssy+v7mtlCM1tmZs+Y2bgs\nr28zs4sLVXhSbNsGc+aEwO9IL5pC2X13mD079OyZORNOOQW2bg3Pzz0XPvOZ0BRUVRVfjUl1yilQ\nWxu+YY0ZA/Pnx12RJFnO0DezbsDNwMnACGCSmY1ottp3gbnuPgqYCDTvl3ADcF/ny02euXND8MfR\ntNNcRUXokXP77aFP/7Bh4Z6ByZND3/K99oq7wuT6xCfCt6gDDoDPfz784RXpCvmc6Y8G1rj7Onff\nAcwBJjRbx4Gm89S9gPdbJ83sVGAdsKLz5SbPrFnwd38X2stLxTnnhPF/+veH666DW29VD51iGDwY\nHn00DJ0xZQo8/3zcFUkS5RP6g4HMMRo3RssyTQO+YmYbgfnA+QBmtgdwKXBVpytNoJUrYdGicJZf\nar1gjjsO1q6Fiy8uvdqSrE+fMP5Sr16hO6dIoeUT+tn+yTfv3zEJuMPdhwDjgF+ZWQUh7G9w9zY7\npJnZZDOrM7O6+vr6fOpOhF/8IpxBf/WrcVcipWTgQPj+90Pbfk1N3NVI0uScGN3MjgSmuftno+eX\nA7j7DzPWWQGc5O4boufrgCOA3wJDo9X6AY3Ale7+s9beLy0To+/YEb7Of+pTYQgDkUw7doTxeRob\nQ3fenj3jrkhKXSEnRl8CHGBmw82sJ+FCbfOpt9cDx0dvfBDQC6h392Pcvcrdq4AbgX9rK/DTZN48\n2LKlNC7gSunp2TPcuPXCC6E/v0ih5Ax9d28ApgALgFWEXjorzGy6mY2PVrsI+LqZLQdmA2d7rq8Q\nKTdrFgwdGi7aiWRz0kmhJ8/06bB5c9zVSFLkbN4ptjQ077z8chgv/3vfg6t0iVvasGZNmA9h0qRw\ngVekNYVs3pECa/rHe845sZYhZWD//eHb3w7zFjz5ZNzVSBIo9Its585w89PYsbq7VfJzxRVhtq2p\nU8OFXZHOUOgX2eLFYTjjs86KuxIpF3vuCT/6ESxZEs74RTpDoV9kNTVhhEXNmiTtceaZ4a7tyy6D\n//u/uKuRcqbQL7KaGjjqKNh777grkXJiBjfdFCax+cEP4q5GyplCv4heeQWefjqMqijSXtXVYcTT\nn/4U/vd/465GypVCv4jui8YZHTeu7fVEWjNjBuyxB1xwQfbJbkRyUegXUU1NuCHr4IPjrkTK1T77\nwLRpYRTUe++NuxopRwr9Itm+HR54IDTtaNRK6YzzzoODDgqjcG7fHnc1Um4U+kXy6KNhflm150tn\n9egR2vXXroUbboi7Gik33eMuIC3mz4fddgvTDop01tixMGECXH11mFC9kN8ee/UKk7gMHZp7XSk/\nGnunSD72MfjoR3ddzBXprHXr4HOfK/xgbNu2QWUl3HMPHHpoYfctXSffsXd0pl8EL7wQfqZOjbsS\nSZL99guzrxXac8+FPybHHgu//nX4RiHJoTb9Imia/UhdNaUcHHxwGNxt5Eg47TT4yU/UPTRJFPpF\nMH9+mPx8v/3irkQkPwMHwsMPw+mnw0UXwbe+BQ0NcVclhaDQ72LbtsEjj6jXjpSf3r1h7ly49FK4\n9dbQ5PPmm3FXJZ2l0O9if/pTmO9UoS/lqKICrrkGbrsNHnwQxowJo8RK+VLod7GamjA07pgxcVci\n0nFf+1roebZ+PRx+OCSwg11qKPS7kHtozz/xxHBDjUg5O+EEeOKJ0I//2GPh7rvjrkg6Ql022+AO\n118PX/pSx25UWb483Dijph1JihEjQs+eCRPgjDPCOP99+8ZdVXJUVcEll3Tteyj027B8efgAHngA\n/vjH9t/12NRVUxOmSJIMGAALF8I3v7nr/3EpjEMPVejHqrY2PN5/f7g7cfz49m1fUxPGQB84sPC1\nicRp993hjjvirkI6Qm36baithcGDw1faCy+Ed9/Nf9stW8LXYN2QJSKlRKHfCnd47LFwwerGG8M4\nJ+0Z0XDBgrAPteeLSClR6Lfi5ZfDRdgxY8KIhqeeGmYteuWV/LavqQkTXlTnHP5IRKR4FPqtaGrP\nP/ro8Hj99eE29Esvzb1tQ0O48HvyyeHmFhGRUqFIasXjj4euaE1TG+63H1x8Mdx5Z3itLU89BX/9\nq9rzRaT05BX6ZnaSma02szVmdlmW1/c1s4VmtszMnjGzcdHysWa21MyejR6PK/QBdJXaWjjqKOjW\nbdeyyy8PF3anToWdO1vftqYmbHfiiV1fp4hIe+QMfTPrBtwMnAyMACaZ2Yhmq30XmOvuo4CJwC3R\n8i3A593948BZwK8KVXhX+utfw5jizYdO2GMPuO46+POf4T//s/Xta2rCtv36dW2dIiLtlc+Z/mhg\njbuvc/cdwByg+bQKDjTdl7cXsAnA3Ze5+6Zo+Qqgl5nt1vmyu9YTT4THbOPlTJwYll9xBWzd2vL1\nDRvgmWfUa0dESlM+oT8Y2JDxfGO0LNM04CtmthGYD5yfZT9nAMvcfXsH6iyq2lro3h0OO6zla2Zw\n002hH/5VV7V8vWk6RLXni0gpyif0sw0+0HwenUnAHe4+BBgH/MrM3t+3mY0ErgW+kfUNzCabWZ2Z\n1dXX1+dXeRd6/PFwO3Tv3tlfHzUKJk+Gn/2s5XR1NTUwbFi4oUtEpNTkE/obgczhxoYQNd9kOBeY\nC+Dui4BeQH8AMxsC3A38o7uvzfYG7j7T3avdvbqysrJ9R1Bg27fD4sW5h0K++mro0wcuuGDXVHLv\nvhvGzz/llPaP0yMiUgz5hP4S4AAzG25mPQkXauc1W2c9cDyAmR1ECP16M+sH1ACXu3uOjo6lYenS\nEPy5Qr9/f5g+PQzGNi/6r/HII/DOO2rPF5HSlTP03b0BmAIsAFYReumsMLPpZtY0BNlFwNfNbDkw\nGzjb3T3abn/ge2b2dPSzT5ccSYE0vymrLd/8Zpg8umlcnvnzw1jjn/50l5YoItJh5iU2zX11dbXX\nxTgtz/jxsHp1+MnHgw+GySWuvjp04zzwQA03KyLFZ2ZL3T3nwC+6IzdDY2PortmeqQ2PPx5OPz00\n9axdq6YdESltCv0Mq1fDG2+0fz7b66/fNcaOumqKSCnTJCoZmtrz2xv6VVVw7bWwaFH4XUSkVOlM\nP0NtbRgOef/927/t1Kkwe3bhaxIRKSSFfoba2tBrR33sRSSpFPqRzZvD7FjtbdoRESknCv1I0xj5\nCn0RSTKFfqS2FnbfPYyrIyKSVAr9SG0tHHEE9OgRdyUiIl1HoQ+89RYsW5bf0AsiIuVMoU+Y07ax\nUe35IpJ8Cn3CRdyKCjjyyLgrERHpWgp9Qnv+IYdA37651xURKWepD/2GhjB8gtrzRSQNUh/6y5fD\n22+rPV9E0iH1od/RQdZERMpR6kP/8cfDROZDhsRdiYhI10t16LuHM32d5YtIWqQ69F98MQy0pou4\nIpIWqQ59teeLSNqkPvT32gtGjoy7EhGR4kh16D/+eGjaqUj1fwURSZPUxt0bb8DKlWraEZF0SW3o\nP/FEeNRFXBFJk9SGfm1tGDv/sMPirkREpHhSHfrV1WG2LBGRtEhl6L/7LtTVqT1fRNInr9A3s5PM\nbLWZrTGzy7K8vq+ZLTSzZWb2jJmNy3jt8mi71Wb22UIW31F1dbBjh0JfRNKne64VzKwbcDMwFtgI\nLDGzee6+MmO17wJz3f3nZjYCmA9URb9PBEYCHwH+ZGYfc/edhT6Q9mi6Keuoo+KsQkSk+PI50x8N\nrHH3de6+A5gDTGi2jgNNU5DsBWyKfp8AzHH37e7+IrAm2l+sVq6EoUOhf/+4KxERKa58Qn8wsCHj\n+cZoWaZpwFfMbCPhLP/8dmxbdJs3w0c+EncVIiLFl0/oW5Zl3uz5JOAOdx8CjAN+ZWYVeW6LmU02\nszozq6uvr8+jpM7ZvBkGDerytxERKTn5hP5GYGjG8yHsar5pci4wF8DdFwG9gP55bou7z3T3anev\nrqyszL/6DlLoi0ha5RP6S4ADzGy4mfUkXJid12yd9cDxAGZ2ECH066P1JprZbmY2HDgAWFyo4jti\n+3b4y18U+iKSTjl777h7g5lNARYA3YDb3X2FmU0H6tx9HnARcJuZXUhovjnb3R1YYWZzgZVAA3Be\n3D13XnstPCr0RSSNcoY+gLvPJ1ygzVx2ZcbvK4Gso9i4+wxgRidqLKjNm8OjQl9E0ih1d+Q2hf7A\ngfHWISISh9SGvs70RSSNUhn6ZrDPPnFXIiJSfKkM/X32ge55Xc0QEUmWVIa+mnZEJK0U+iIiKaLQ\nFxFJkVSF/s6d4eYshb6IpFWqQr++HhobFfoikl6pCn310ReRtFPoi4ikSKpC/9VXw6NCX0TSKlWh\nr3F3RCTtUhf6/fpBr15xVyIiEo/Uhb6adkQkzRT6IiIpotAXEUmR1IS+u0JfRCQ1ob91a5gUXaEv\nImmWmtDXjVkiIgp9EZFUUeiLiKSIQl9EJEVSFfq9e8Oee8ZdiYhIfFIV+oMGgVnclYiIxCc1of/q\nqxpoTUQkNaGvG7NERPIMfTM7ycxWm9kaM7ssy+s3mNnT0c/zZrY147UfmdkKM1tlZjeZxdPAotAX\nEYHuuVYws27AzcBYYCOwxMzmufvKpnXc/cKM9c8HRkW/HwUcDRwSvVwLfAp4uED15+Wdd+DNNxX6\nIiL5nOmPBta4+zp33wHMASa0sf4kYHb0uwO9gJ7AbkAP4LWOl9sx6q4pIhLkE/qDgQ0ZzzdGy1ow\ns2HAcOAhAHdfBCwENkc/C9x9VWcK7giFvohIkE/oZ2uD91bWnQjc5e47Acxsf+AgYAjhD8VxZnZs\nizcwm2xmdWZWV19fn1/l7aDQFxEJ8gn9jcDQjOdDgE2trDuRXU07AKcBT7r7NnffBtwHHNF8I3ef\n6e7V7l5dWVmZX+XtoNAXEQnyCf0lwAFmNtzMehKCfV7zlczsQGBvYFHG4vXAp8ysu5n1IFzEjaV5\np3t3+PCHi/3OIiKlJWfou3sDMAVYQAjsue6+wsymm9n4jFUnAXPcPbPp5y5gLfAssBxY7u73FKz6\nPG3eHG7MqkjNXQkiItnl7LIJ4O7zgfnNll3Z7Pm0LNvtBL7RifoKQn30RUSCVJz7KvRFRILUhL7G\n3RERSUHov/cebNmiM30REUhB6L8W3f+r0BcRSUHoq4++iMguCn0RkRRR6IuIpEgqQt8MBgyIuxIR\nkfilIvT794cePeKuREQkfqkIfTXtiIgECn0RkRRR6IuIpEiiQ7+xMdycpdAXEQkSHfpbtkBDg8bd\nERFpkujQVx99EZEPSnTov/pqeFToi4gEiQ59nemLiHyQQl9EJEUSH/p9+0Lv3nFXIiJSGhIf+jrL\nFxHZRaEvIpIiCn0RkRRJbOi7K/RFRJpLbOi/+Sb87W8KfRGRTIkNfXXXFBFpKfGhr3F3RER2SXzo\n60xfRGSXvELfzE4ys9VmtsbMLsvy+g1m9nT087yZbc14bV8zu9/MVpnZSjOrKlz5rVPoi4i01D3X\nCmbWDbgZGAtsBJaY2Tx3X9m0jrtfmLH++cCojF38FzDD3R8wsz5AY6GKb8vmzbDbbtCvXzHeTUSk\nPORzpj8aWOPu69x9BzAHmNDG+pOA2QBmNgLo7u4PALj7Nnd/p5M15+XVV8NZvlkx3k1EpDzkE/qD\ngQ0ZzzdGy1ows2HAcOChaNHHgK1m9jszW2Zm10XfHLqc+uiLiLSUT+hnO1f2VtadCNzl7juj592B\nY4CLgcOA/YCzW7yB2WQzqzOzuvr6+jxKyk2hLyLSUj6hvxEYmvF8CLCplXUnEjXtZGy7LGoaagB+\nD3yy+UbuPtPdq929urKyMr/Kc1Doi4i0lE/oLwEOMLPhZtaTEOzzmq9kZgcCewOLmm27t5k1Jflx\nwMrm2xba3/4GW7cq9EVEmssZ+tEZ+hRgAbAKmOvuK8xsupmNz1h1EjDH3T1j252Epp0HzexZQlPR\nbYU8gGw0TaKISHY5u2wCuPt8YH6zZVc2ez6tlW0fAA7pYH0doj76IiLZJfKOXIW+iEh2iQ59jbsj\nIvJBiQ39igooUEcgEZHESGzoDxgA3YpyG5iISPlIbOirPV9EpCWFvohIiiQy9JsGWxMRkQ9KXOg3\nNMDrryv0RUSySVzov/46uCv0RUSySVzo68YsEZHWKfRFRFJEoS8ikiKJDX0NwSAi0lIiQ//DH4ae\nPeOuRESk9CQy9HWWLyKSXSJDX+35IiLZKfRFRFIkUaHvriEYRETakqjQf+MNeO89hb6ISGsSFfqa\nEF1EpG2JCn3dmCUi0jaFvohIiij0RURSJHGh36dP+BERkZYSF/o6yxcRaZ1CX0QkRRIX+hp3R0Sk\ndYkLfZ3pi4i0Lq/QN7OTzGy1ma0xs8uyvH6DmT0d/TxvZlubvd7XzF4xs58VqvDm3noL3n5boS8i\n0pbuuVYws27AzcBYYCOwxMzmufvKpnXc/cKM9c8HRjXbzQ+ARwpScSu2b4eJE2FU83cWEZH35Qx9\nYDSwxt3XAZjZHGACsLKV9ScB3296YmaHAgOAPwLVnaq2Df37w+zZXbV3EZFkyKd5ZzCwIeP5xmhZ\nC2Y2DBgOPBQ9rwCuBy5p6w3MbLKZ1ZlZXX19fT51i4hIB+QT+pZlmbey7kTgLnffGT3/FjDf3Te0\nsn7YmftMd6929+rKyso8ShIRkY7Ip3lnIzA04/kQYFMr604Ezst4fiRwjJl9C+gD9DSzbe7e4mKw\niIh0vXxCfwlwgJkNB14hBPuXm69kZgcCewOLmpa5+5kZr58NVCvwRUTik7N5x90bgCnAAmAVMNfd\nV5jZdDMbn7HqJGCOu7fW9CMiIjGzUsvo6upqr6uri7sMEZGyYmZL3T1nD8lE3ZErIiJtU+iLiKRI\nyTXvmFk98HIndtEf2FKgckpB0o4HkndMSTseSN4xJe14oOUxDXP3nH3eSy70O8vM6vJp1yoXSTse\nSN4xJe14IHnHlLTjgY4fk5p3RERSRKEvIpIiSQz9mXEXUGBJOx5I3jEl7XggeceUtOOBDh5T4tr0\nRUSkdUk80xcRkVYkJvRzze5VjszsJTN7NpqRrOxuUzaz283sdTN7LmPZh8zsATN7IXrcO84a26uV\nY5oWzQzXNHvcuDhrbA8zG2pmC81slZmtMLN/iZaX5efUxvGU82fUy8wWm9ny6JiuipYPN7Onos/o\nN2bWM6/9JaF5J5rd63kyZvcCJmXO7lWOzOwlwiB1Zdm/2MyOBbYB/+XuB0fLfgT8xd2vif447+3u\nl8ZZZ3u0ckzTgG3u/uM4a+sIMxsEDHL3P5vZnsBS4FTgbMrwc2rjeP6B8v2MDNjD3beZWQ+gFvgX\n4NvA79x9jpndCix395/n2l9SzvTfn93L3XcATbN7SYzc/VHgL80WTwB+Gf3+S8I/yLLRyjGVLXff\n7O5/jn5/izCo4mDK9HNq43jKlgfboqc9oh8HjgPuipbn/RklJfTznt2rzDhwv5ktNbPJcRdTIAPc\nfTOEf6DAPjHXUyhTzOyZqPmnLJpCmjOzKsL81k+RgM+p2fFAGX9GZtbNzJ4GXgceANYCW6NRkKEd\nmZeU0G/P7F7l5Gh3/yRwMnBe1LQgpefnwEeBvwc2E6YILStm1gf4LXCBu78Zdz2dleV4yvozcved\n7v73hEmsRgMHZVstn30lJfTbM7tX2XD3TdHj68DdhA+73L0Wtbs2tb++HnM9nebur0X/KBuB2yiz\nzylqJ/4tcKe7/y5aXLafU7bjKffPqIm7bwUeBo4A+plZ00RYeWdeUkL//dm9oivYE4F5MdfUKWa2\nR3QhCjPbAzgReK7trcrCPOCs6PezgD/EWEtBNIVj5DTK6HOKLhL+Aljl7j/JeKksP6fWjqfMP6NK\nM+sX/b47cALhWsVC4AvRanl/RonovQMQdcG6EegG3O7uM2IuqVPMbD/C2T2EaS1/XW7HZGazgU8T\nRgN8Dfg+8HtgLrAvsB74oruXzYXRVo7p04RmAwdeAr7R1B5e6sxsDPAY8CzQGC2+gtAOXnafUxvH\nM4ny/YwOIVyo7UY4UZ/r7tOjjJgDfAhYBnzF3bfn3F9SQl9ERHJLSvOOiIjkQaEvIpIiCn0RkRRR\n6IuIpIhCX0QkRRT6IiIpotAXEUkRhb6ISIr8P5Pj1GPv5v0tAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7faef39270b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "ind_max = scores.index(max(scores))\n",
    "ax.plot(scores, 'b', ind_max, max(scores), 'ro')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Melhor valor de k:  21\n"
     ]
    }
   ],
   "source": [
    "best_k = scores.index(max(scores))*4 + 1\n",
    "print(\"Melhor valor de k: \", best_k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.92      0.75      0.83        65\n",
      "          1       0.57      0.84      0.68        25\n",
      "\n",
      "avg / total       0.83      0.78      0.79        90\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# relatório de classificação para os dados de validação\n",
    "knn.fit(X_train, y_train, k=best_k, distance='euclidean')\n",
    "y_pred = knn.predict(X_valid)\n",
    "print(classification_report(y_pred, y_valid))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
